Empirical ad pod detection

ABSTRACT

A system and method for analyzing the viewing behavior of end users that are viewing video content to determine the likely presence of one or more advertisements being viewed and to assess the viewers who stop viewing during the course of such advertisements. The system periodically or continuously receives tune data reflecting the viewing behavior of end users, analyzes the cumulative viewing behavior reflected by the received tune data, and delivers a report of the analysis to advertisers, agencies, media sellers, or other parties that are interested in measuring the effectiveness of advertisements on end users. The analysis by the system can be done over multiple different types of content-distribution platforms.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit of U.S. Provisional PatentApplication No. 61/612,813, entitled “EMPIRICAL AD POD DETECTION,” filedMar. 19, 2012, which is incorporated herein by reference in itsentirety.

BACKGROUND

The shift to digital broadcasting has improved the ability of contentproviders to determine the viewing behavior of their subscribers andother end users. Data received through digital converters, or set topboxes (STBs), can be collected and compared with a programmed scheduleof broadcast content in order to estimate which programs are beingviewed. The resulting viewing data can be communicated to a contentprovider, advertisers, and other interested parties to indicate when aparticular group of subscribers has tuned to a particular piece ofcontent.

With the advent of new distribution channels and flexible programmingvia which content providers are able to present content, however, it hasbecome increasingly difficult using traditional techniques to determinethe number of viewers viewing particular content and the impact ofadvertisements shown during that particular content. Although digitalviewing data can be analyzed against regularly scheduled programming inwhich both program and advertisement times are scheduled, suchtraditional techniques fail for other types of programming. For example,local television stations will sometimes deviate from a programmedschedule of broadcasts, thereby making it difficult to assess whichcontent is being viewed at which time. As another example, liveprogramming will often have unpredictable schedules, such as sportingevents which vary in length or important news events which pre-emptregularly-scheduled programming. Live programming therefore oftenresults in inconsistent commercial breaks, length of programming, andstart times for subsequent programming presented by a content provider.As yet another example, the use of digital video recorders (DVRs) allowsviewers to watch recorded and/or streamed content on their own schedule.Accordingly, there is a need to improve the ability to assess a numberof content viewers, particularly for content presented by contentproviders in a non-scheduled fashion.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an environment in which a viewing behavior analysissystem operates for detecting sets of advertisements (“ad pods”) inpresented content.

FIG. 2 is a flow diagram of a process performed by the viewing behavioranalysis system to perform improved tracking of viewer behavior bydetecting ad pods.

FIG. 3A is a representative graph of the aggregate viewership of asingle channel of content.

FIG. 3B is a representative graph of the rate of change of the channelin FIG. 3A that is used by the system to predict the occurrence of adpods in the content.

FIG. 4 is an exemplary graph of ad pods detected for a program of livebroadcast content over the presentation period of the program.

FIG. 5 is an exemplary report generated by the viewing behavior analysissystem.

DETAILED DESCRIPTION

A system and method for analyzing the viewing behavior of end users thatare viewing video content to determine the likely presence of one ormore advertisements being viewed (“ad pods”) and to assess the viewerswho stop viewing the content during the course of such advertisements isdisclosed. The system periodically or continuously receives tune datareflecting the viewing behavior of end users, analyzes the cumulativeviewing behavior reflected by the received tune data, and delivers areport of the analysis to advertisers, agencies, media sellers, or otherparties that are interested in measuring the effectiveness ofadvertisements on end users.

The system receives data reflecting the viewing behavior of end usersfrom a client incorporated in or coupled to an end user's video playbackdevice, such as a set-top box (STB), digital video recorder (DVR),satellite TV receiver, internet protocol television (IPTV), computer,smartphone, or other viewer measurement devices and/or equipment thatenable an end user to view content over a network. The clients monitorand capture events reflecting the viewing of content by end users, forexample by recording channel-change events or other indications ofviewing behavior. The system analyzes the data reflecting the viewingbehaviors tracked by clients in order to determine when users arewatching a particular channel or content, or when users have likelystopped watching a particular channel or content. The system aggregatesthe viewing behavior across multiple clients to determine the totalnumber of end users likely watching particular content, such as contentpresented on a particular channel at a particular day and time. Theanalysis is based on one or more tune events occurring during a selectedtime period for which the data is being analyzed.

The aggregation by the system can be done over multiplecontent-distribution platforms, each capable of supporting clients. Theviewing behavior can be analyzed over a period of time or duringreal-time, such as at the same time that particular content is beingpresented by a content provider. The content can include, for example,content broadcast on a particular channel on a particular day and at aspecified time, streamed through a content provider's network, orre-played through a STB device.

The aggregate viewing behavior data (or “tune data”) is analyzed by thesystem to generate a report of when significant drops in viewershipoccurred, indicating the likely presence of an ad pod. The report canhighlight those ads and content where the drops occurred more or lessfrequently. The report is made available to advertisers, agencies, mediasellers, or other parties that are interested in measuring theeffectiveness of advertisements on end users.

Various embodiments of the invention will now be described. Thefollowing description provides specific details for a thoroughunderstanding and an enabling description of these embodiments. Oneskilled in the art will understand, however, that the invention may bepracticed without many of these details or with variations which are notspecified here but which follow from the description in a way that willbe clear to one skilled in the art. Additionally, some well-knownstructures or functions may not be shown or described in detail, so asto avoid unnecessarily obscuring the relevant description of the variousembodiments. The terminology used in the description presented below isintended to be interpreted in its broadest reasonable manner, eventhough it is being used in conjunction with a detailed description ofcertain specific embodiments of the invention.

FIG. 1 and the following discussion provide a general description of asuitable computing environment in which a viewing behavior analysissystem (“system”) 100 is implemented. The system 100 is configured toreceive tune data from one or more content providers 124. In therepresentative environment, the depicted content provider 124 is a cabletelevision operator that transmits program signals on a cable 122 (e.g.,coaxial, optical). Content providers also include, but are not limitedto, satellite television operators, local or regional broadcasters, anddistributors of content over other transmission media such as theInternet or other network. Subscribers and end users view the programsignals on televisions, video monitors, or other audio/video playbackdevices 120 (each of which is referred to herein as a “video playbackdevice”). Each of the video playback devices is associated with acorresponding internal tuner or external set top box (STB) 118 thatserves as an interface between the video playback device and the cable122 or other transmission means on which the television program or otheraudio-video content is received. In some embodiments, the set top boxes118 comprise cable television converters or satellite dish receivers.However, the set top boxes could also be digital video recorders (DVR),gaming consoles, audience measurement meters, or other electroniccomponents which either allow a user to tune to a desired audio/videostream, or provide the functionality of recording tune events for lateranalysis. Broadly stated, the phrase “set top box” is used herein torefer to any device, component, module, or routine that enables tunedata to be collected from an associated video playback device. Set topboxes may be stand-alone devices or set top box functionality may beincorporated into video playback devices.

The content providers 124 may also directly transmit content to othervideo playback devices via network 110. For example, video playbackdevices may also include smartphones 112, computers 114, and tablets116. Content may be directly streamed to the video playback devices fromcontent provider 124, or indirectly via devices such as digital mediareceivers (DMRs) 128. As used herein, a “video playback device”therefore also encompasses computers, smartphones, laptops, tablets, orother computing devices capable of displaying content provided by acontent provider.

The content provider 124 receives signals indicative of tuning and otherevents that occur on video playback devices. The tuning events canrepresent such things as channel changes, recording or replaying ofcontent that was transmitted to a STB, and changes in playback ofcontent, such as when a subscriber pauses, fast forwards, or rewinds aprogram or otherwise varies its normal playback. In addition, the tuningevents may indicate when a subscriber requests information from aninteractive television subscription service.

The tune data is collected for the content provider 124 by clients 126that are incorporated in or are associated with set top boxes 118 orvideo playback devices. A “client” refers to a software or hardwaremodule within a device that is capable of monitoring, collecting,storing, and communicating tune data from a video playback device. Thetune data is communicated to a content provider and/or third party forfurther analysis. The type of client 126 depends on the particular videoplayback device in which it is incorporated. As one example, a client126 may be integrated in a set top box 118. The STB may be coupled to amonitor or other display, for example a STB 118 coupled via a coaxialcable to an analog television, or a STB 118 coupled via an HDMI or otherdigital signal channel to a digital television or other digital videodisplay equipment (e.g., a computer monitor, video projector, etc.). TheSTB may include a digital video recorder (“DVR”) that is configured torecord one or more live video broadcast streams and is capable ofproviding deferred viewing or “time shifting” of a particular livebroadcast that occurred at a certain time. As another example, a clientmay be directly built into a device having a monitor or display, such asa television 120 or a computer 114. For example, a client may besoftware in a personal computer 114 with a television tuning cardcapable of receiving a television broadcast over the air or over a cabletelevision network. In another example, a personal computer 114 may havea client and receive a television broadcast over the Internet, e.g., viaInternet, web television, IPTV, or similar streaming mechanism. In yetanother example, a client may be incorporated in a mobile device, suchas a mobile smartphone 112, that receives television over a wirelessnetwork.

In some embodiments, a client 126 may be integrated into equipment (notshown) serving multiple end users, such as head-end equipment of a cabletelevision system, or head-end equipment of an internet communicationnetwork, or a single master antenna television (“SMATV”) head-end thatreceives and rebroadcasts satellite television signals throughout aproperty.

Each client 126 records tune data associated with the associated videoplayback device. The client tracks or monitors the viewing behavior of aviewer using a method that is appropriate to the device in which theclient is incorporated. For example, a STB client may determine that oneor more viewers are likely viewing a video broadcast if the STB hasrecently received a channel, volume, or other input from the viewer viathe remote control. In another embodiment, a STB client can determinethat a particular viewer associated with that STB is likely viewing avideo broadcast if the STB is turned on. In yet another embodiment, aDVR client can determine that a viewer is likely watching a videobroadcast if the viewer provides a command to play back video contentfrom a previously recorded video broadcast. Events reflecting viewingbehavior that are tracked by a client may include, but are not limitedto: a command to power-up or power-down a corresponding monitor ordisplay, a channel or content switching event, such as channel changingevents (e.g., “channel up”, “channel down”, “switch to channel 5”,“switch to channel 13.2” (i.e., main channel 13, sub-channel 2), “accesspay per view content channel 20”, etc.) with or without the use of anelectronic program guide, or commands controlling the playback ofprerecorded content (e.g., “play”, “stop”, “fast forward”, and “rewind”for content prerecorded on a DVR device, etc.). Each event is recordedby the client with an associated date/time stamp reflecting when theevent occurred. To store tune data, for each event the client 126 mayinclude a unique identifier associated with the set top box or videoplayback device, an identifier of the tune action (e.g., channel change,play, pause, etc.), an identifier that directly or indirectly identifiescontent being viewed (e.g., a channel that was tuned, or a uniqueidentifier associated with the tuned content), and a time and data stampindicating when the tuning event occurred. It will be appreciated that agreater or lesser amount of tune data for each event may be captured bya client.

Tune data is transmitted from clients 126 to content providers 124and/or third party aggregators of tune data. A content provider or thirdparty-aggregator may then periodically or continuously provide the tunedata to the viewing behavior analysis system 100. Alternatively, theclients 126 may transmit the tune data directly to the viewing behavioranalysis system 100. In some embodiments, the tune data is continuouslytransmitted by the client 126 to the intended recipient. For example,the client may continuously report tune data in cases where the clienthas an available and robust network connection, such as when a client ison an IPTV network. In such circumstances, the client need not cache(i.e., store) data reflecting the tracked viewing behavior on the clientdevice. In some embodiments, the tune data is periodically transmittedby the client 126 to the intended recipient. For example, the client maysend tune data at predetermined intervals to the intended recipient. Forexample, the client may transfer the cached data to the system 100 orintended recipient every 6 hours, every time the associated STB ispowered “on,” every evening at 3 am, etc. Between transmissions, theclient caches the tune data reflecting the tracked viewing behavior.When the client caches the tune data, the client employs local storagesuch as random access memory, a mass storage device (such as the massstorage device used by DVRs), flash drive storage, or other storagedevices known in the art.

In some embodiments, a client may only transmit the data reflecting thetracked viewing behavior to the intended recipient when the intendedrecipient requests the client to deliver the data (i.e., a “pull”transaction). In some embodiments, the client can decide when to sendthe data reflecting the tracked viewing behavior to the intendedrecipient (i.e., a “push” transaction). For example, the client maytransmit the tune data any time the client has new data available thathas not yet been transmitted. In another example, the client may cachethe tracked tune data in a buffer, and transmit the data when the cachebuffer is becoming full to minimize the number of times the client musttransmit data.

In some embodiments, the client may transmit all cached tune data at aspecific and reoccurring time, such as every day at 2:00 AM, everyMonday at midnight, once per month on the 26th day of the month, or onceper year on every June 26th, and so on. In other embodiments, apredetermined event may trigger the transmission of tune data. Forexample, when the client device is powered on or off, the device'sbooting or shutdown procedure can trigger the transmission.

The viewing behavior analysis system 100 receives tune data from contentproviders 124, from third-party aggregators of tune data (not shown),and/or directly from clients 126. The tune data can be received overnetworks 110, such as public or private networks, and wired or wirelessnetworks, including the Internet. Alternatively, on a periodic basis,the tune data may be provided to the system 100 by a content provider orthird-party aggregator on a computer readable media such as tape drive,DVD, CD-ROM, flash drive, mechanical or solid state hard drive, etc. Thesystem 100 stores the received tune data in a viewer database 108.

The system 100 includes a non-transitory computer readable medium 102 onwhich computer readable instructions are encoded for performing ananalysis of the tune data. The computer readable medium is coupled to aprocessor 104, which executes the stored instructions in order toimplement the functionality disclosed herein. In particular, the system100 includes instructions specifying an analysis module 106 and areporting module 107. The analysis module 106 analyzes the tune data toidentify ad pods within viewership data. The analysis module uses theidentified ad pods to improve the accuracy of various viewershipcalculations. The analysis module stores the data identifying the adpods in ad pod database 109. The report module 107 generates reports ofthe analysis and provides the report to advertisers, agencies, mediasellers, or other parties that are interested in measuring theeffectiveness or other aspects or metrics of advertisements. Thefunctionality of each module will be further described with respect toFIGS. 2-5.

One skilled in the art will understand that aspects and implementationsof the system have been described in the general context ofcomputer-executable instructions that are executed on a server computer.It will be appreciated that the disclosed technology can be practicedwith other computer system configurations, including desktops, laptops,multiprocessor systems, microprocessor-based systems, minicomputers,mainframe computers, or the like. The disclosed technology can beembodied in a special purpose computer or data processor that isspecifically programmed, configured, or constructed to perform one ormore of the computer-executable instructions explained in detail below.

The terms “computer” and “computing device,” as used generally herein,refer to devices that have any data processor and non-transitory memorycontaining instructions that are executed by the processor. Dataprocessors include programmable general-purpose or special-purposemicroprocessors, programmable controllers, application-specificintegrated circuits (ASICs), programmable logic devices (PLDs), or thelike, or a combination of such devices. Software may be stored inmemory, such as random access memory (RAM), read-only memory (ROM),flash memory, or the like, or a combination of such components. Softwaremay also be stored in one or more storage devices, such as magnetic oroptical-based disks, flash memory devices, or any other type ofnon-volatile storage medium or non-transitory medium for data. Softwaremay include one or more program modules, which include routines,programs, objects, components, data structures, and so on that performparticular tasks or implement particular abstract data types.

The disclosed technology can also be practiced in distributed computingenvironments, where tasks or modules are performed by remote processingdevices, which are linked through a communications network, such as aLocal Area Network (“LAN”), Wide Area Network (“WAN”), or the Internet.In a distributed computing environment, program modules or subroutinesmay be located in both local and remote memory storage devices. Aspectsof the technology described herein may be stored or distributed ontangible, non-transitory computer-readable media, including magnetic andoptically readable and removable computer discs, stored in firmware inchips (e.g., EEPROM chips). Alternatively, aspects of the invention maybe distributed electronically over the Internet or over other networks(including wireless networks). Those skilled in the relevant art willrecognize that portions of the system may reside on a server computer,while corresponding portions reside on a client computer incommunication with the server computer.

FIG. 2 is a flowchart of a process 200 of analyzing viewer behavior thatis implemented by the system 100 in order to identify ad pods and assessthe effect of ad pods on viewing behavior. An “ad pod” refers to one ormore advertisements, such as a singular advertisement or a group ofadvertisements that is presented during content broadcast by a contentprovider. An ad pod can include visual and/or audio content, and mayinclude national or local insertions.

The system aggregates the viewing behavior across multiple clients todetermine the total number of end users likely watching particularcontent, such as content broadcast on a particular channel at aparticular day and time. The analysis may include end users which viewedthe content at the time of broadcast as well as end users who viewed thecontent on a delayed basis using, for example, a DVR or other delayedstreaming functionality. At a block 202, the system 100 selects clienttune data corresponding to desired clients and a desired time period onwhich to perform an analysis. As described with respect to FIG. 1, thesystem receives tune data from clients in multiple content-deliverychannels. To analyze the tune data, a system operator thereforeinitially selects a desired set of clients on which the analysis will beperformed. The analysis may be performed on all clients, clientsassociated with certain or all distribution channels, clients associatedwith certain or all content providers, clients associated with differentplayback devices, clients associated with certain or all viewercharacteristics such as demographics or behavioral tendencies, etc. Inaddition, the system operator also selects a desired period of time overwhich to analyze the tune data. In some embodiments, the system 100analyzes tune data from a short period immediately surrounding thepresentation of content. For example, the system may immediately collectand analyze tune data within a short period following the end of thepresented content. In some embodiments, the system 100 analyzes tunedata over a longer time frame. For example, the system can analyze tunedata surrounding the time that a television program was initiallybroadcast through a broadband cable, as well as additional tune data inthe subsequent week representing times when the same television programwas replayed through a DVR 118 or a digital media receiver (DMR) 128.The tune data can be analyzed for a selected time frame, such as a week,a month, etc.

At block 204, the system operator selects a specific distributionchannel and time frame, or a specific program or content (the specifiedprogram or content having a corresponding time frame), on which toperform the viewership analysis. The system then constructs a viewershipprofile for the selected content and selected time period from the tunedata collected from each client. FIG. 3A is a representative graph 400of a viewership profile 402. The profile 402 represents an aggregatenumber of viewers or screens of specified content over a selected timeperiod. The y-axis of the graph represents the number of viewers orscreens that are displaying the particular channel. In the depictedexample, the number of viewers or screens is shown between 48,000 and58,000. The x-axis of the graph represents the elapsed time, starting inthe depicted example at 9:00 pm and extending until 9:30 pm. The plottedviewership profile line on the graph represents the number of viewers orscreens at each particular time.

The aggregate number of viewers or screens that is used by the system,such as the aggregate viewers displayed in FIG. 3A, may be an actualnumber or an estimated number of total viewers or screens. The systemmay employ a variety of techniques to estimate the number of totalviewers of particular content or screens displaying particular content.In some embodiments, the system may extrapolate from a specific numberof clients for which information is available in order to infer thelikely behavior of a larger audience. For example, if a cable televisionnetwork contains 1,000,000 STBs, but only 30,000 of those STBs containclients capable of tracking and sending tune data to the system, thesystem may extrapolate from the limited number of STBs from which datais received. For example, if an audience of 634 members were watchingparticular content on those 30,000 STBs, the system may project that21,133 members ((1,000,000/30,000)×634) of the larger audience werelikely watching the particular content on their televisions. To arriveat the more accurate estimate of the audience, the system merelylinearly scales the known viewers in a manner proportional to theoverall population of the content distribution network.

In some embodiments, to improve the calculation of the aggregate numberof viewers or screens, more weight may be assigned by the system 100 todata received from certain clients over others. For example, a clienttracking viewing behavior of a television in a sports bar (likelywatched by multiple viewers) may be given more weight than a clienttracking viewing behavior on the display of a mobile phone (likelywatched by a single viewer). In other examples, demographic informationabout the client end users may be employed to weight the analysis anddetermine a number of likely viewers watching each screen. Techniquesthat may be employed to determine the aggregate number of viewers orscreens is described further in U.S. patent application Ser. No.13/081,437, filed Apr. 6, 2011, entitled “METHOD AND SYSTEM FORDETECTING NON-POWERED VIDEO PLAYBACK DEVICES,” the content of which ishereby incorporated by reference in its entirety. To take into account agreater reliability of certain data over others, the system 100 maymultiply viewership numbers from one set of clients by a factor greaterthan one (e.g., multiplying by 1.05) while multiplying viewershipnumbers from another set of clients by a factor less than one (e.g.,multiplying by 0.95).

Using the data reflecting aggregate viewing behavior, the system 100determines or estimates when advertisements are displayed during a pieceof viewed content and calculates the number of viewers or screens thatare retained during those advertisements. Sharp drops in the number ofviewers around a particular time on a particular channel can indicatethat the programming has been interrupted with an advertisement. Forexample, FIG. 3A illustrates sharp drops in the number of screens atthree times during the represented timeframe—around 9:03 pm, around 9:13pm, and around 9:24 pm. The profile 402 also contains more gradualdeclines, such as between 9:01 and 9:03 pm. The sharper drops noted byvertical lines 406, 408, 410, can signify an ad pod, whereas the moregradual declines can signify a loss in viewer interest.

Returning to FIG. 2, at block 206 the system calculates a rate of changefor the viewership profile 402. The system 100 analyzes the rate ofchange of each drop in the profile in order to determine whether eachdrop is a result of advertisements or a result of a natural declinecaused by viewers becoming uninterested in the broadcast content. If therate of change of the drop (i.e., the slope of the viewership line onthe graph) exceeds a predetermined threshold, the system interprets thedrop to be caused by an ad pod. Calculating a derivative of theviewership profile makes it easier for the system to identify the mostsignificant valleys, or drops, occurring during the selected timeperiod.

FIG. 3B is a representative graph 412 illustrating a rate of change inthe number of screens, or viewers, of the viewership profile 402. Thesystem calculates a derivative of the viewership profile 402 and plotsthe derivative as a rate of change profile 414 on the graph. The y-axisof the graph represents a rate of change in a number of screens that aredisplaying on the particular channel. In the depicted example, the rateof change varies from approximately +100 screens/second to approximately−100 screens/second. The x-axis of the graph represents elapsed time,starting in the depicted example at 9:00 pm and extending until 9:30 pm.The three sharp drops observed in FIG. 3B correspond to the maximumnegative measured rate of change of screens at those correspondingtimes. The change in the number of screens is calculated by the systemfrom the number of screens data, for example by subtracting the numberof viewers at a given point of time from the number of screens at apreceding point in time. For example, the system subtracts the number ofscreens at time=9:25 PM (e.g., 50,000) from the number of screens attime=9:24 PM (e.g., 53,800). The system then calculates the rate ofchange in the number of screens for the corresponding unit of elapsedtime as follows: (50,000−53,800)/1 minute=−3,800 screens/minute or −63screens per second. The calculated loss of the number of screens betweenthe time period of 9:24 PM and 9:25 PM for the depicted data set istherefore an average of 63 screens lost (or −63 screens gained) persecond.

The aforementioned calculation may be performed by the system 100 forany time increment in the specified time range. For example, the rate ofchange of the number of screens displayed can be determined for eachminute between 9:00 PM and 9:30 PM. A variety of time increments can beemployed for gathering data or for the analysis, e.g., 1 second, 30seconds, 1 minute, and so on. However, it is preferable for the utilizedtime increment to be sufficiently small in order to allow multiplesamples to fall within the period during which advertisements are beingdisplayed, and to allow adjacent samples to fall within a period duringwhich normal (i.e., non-advertisement) program content is being viewed.It will be understood that references herein to “screens” is equallyapplied to other units of viewer measurement, such as viewers, set-topboxes, computers, mobile devices, and so forth.

Returning to FIG. 2, at blocks 208-214 the system 100 performs ananalysis to identify the number of ad pods within the selected timeperiod and to estimate the start and end of each ad pod. Variousmethodologies may be used by the system to identify the start of adpods. In one methodology, at block 208 the system 100 identifies thestart of an ad pod by identifying when the rate of change of the numberof screens exceeds a threshold rate. For example, the system mayestimate that the start of an ad pod is indicated when the rate ofchange of screens on the rate of change profile 414 shows a decline inexcess of 50 screens per minute. The threshold rate can be calculatedfor the content being analyzed based on a percentage of the tune dataand the number of screens, or viewers, at the start of the selected timeperiod. The threshold can be calculated by an algorithm or by a user inorder to give the most sensible results based on the anticipated numberof ad pods for content of a given length. In some embodiments, thethreshold is selected by calculating a mean rate from a ranked order ofall potential ad pod rates.

At a decision block 210, the system determines if there is additionaltime remaining to be analyzed in the selected time period. The system100 starts at the beginning of the selected time period and processesthrough to the end of the time period to identify the start point of allad pods in the time period. If additional time remains to be analyzed,processing returns to block 208. Otherwise, processing continues toblock 212.

In another methodology to identify the start of ad pods, the system 100may compare and rank all spikes in rates of change during the selectedtime period, and choose the top-ranked rates of change to represent thestart of that many distinct ad pods. Because numerous drops typicallyoccur during a selected time period, the system may filter to onlyidentify the most significant drops. The system may choose a number oftop-ranked rates of change that is equal to an anticipated number of adpods or a user-specified number of ad pods. For example, the system canbe configured to identify the three (3) most significant drops over ahalf-hour period to be ad pods. The anticipated number of ad pods mayvary based on the length of time or type of content being analyzed. Thatis, the negative “spikes” in the derivative graph may be ranked in orderof their magnitude, and the N most-negative spikes (sufficientlyseparated by normal viewing) can be selected as the start of N ad pods.N may be chosen to be a known or estimated “typical” number of ad podsfor the duration of the content. For example, it may be known that for ahalf-hour program, the particular network or content provider alwaysshows three sets of ads, in which case N=3 and the three most-negativeperiods of the time derivative of the viewership metric are thenidentified as the start of the ad pods.

After identification of ad pod starts, at block 212 the system 100estimates the end of each identified ad pod. In one methodology toestimate an end of an ad pod, the system may use a historical averagelength of an ad pod in order to estimate the end. For example, sincecommercial breaks on broadcast television are typically two minutes inlength, the system may use a two minute period as the default period tocalculate the end of the ad pod. The historical average length that isused may vary depending on type of content (e.g., sports event, news,sitcom, serial), time of day (e.g., prime time, early morning), or otherfactor.

In another methodology to identify the end of ad pods, the system 100estimates the end of each ad pod by assessing when the rate of change ofthe number of screens falls below a second threshold rate. When the rateof change of the number of viewers or screens rebounds sufficiently inthe positive direction, it may indicate that the advertisements havebeen completed and the content programming is resuming. An end of an adpod may be indicated when the rate of screen loss is declining or therate of screen loss shows an increase in screens. For example, using theprevious example the system may consider an ad pod to have ended whenthe rate of change falls below 25 screens per minute. The secondthreshold rate may be the same as or different than the threshold rateused to estimate when an ad pod has started.

It will be appreciated by one skilled in the art that the system 100 mayuse other derivations to calculate the start or end of an ad pod. Forexample, the system may determine a change in the rate of change of theviewers (i.e., a second derivative of the number of viewers with respectto time), or a change in the change of the rate of change of the viewers(i.e., a third derivative of the number of viewers with respect to time)and so on. The system may determine that a certain event is likelyoccurring, such as the beginning of an ad pod or a change of content,when any one of the aforementioned derivatives exceeds a predeterminedvalue.

At block 214, the system 100 may implement various reconciliationpolicies to avoid arriving at erroneous ad pod estimations. For example,one rule that the system may implement is that two ad pods should notoccur in an overlapping fashion. If the system 100 identifies two adpods that appear to overlap in time such that the end of the first adpod occurs later in time than the start of the second ad pod, the systemmay combine the two ad pods into a single ad pod having a length equalto the sum of the non-overlapping portions of the individual ad pods.Alternatively, the system may discard the second overlapping ad pod butkeep the first ad pod, on the assumption that the second ad pod is dueto an error in interpreting the tune data.

After block 214, the system 100 will have identified and recorded thenumber of ad pods within the time period, and estimated the start timeand the end time of each ad pod. The identified ad pods within the timeperiod are stored by the system in ad pod database 109 for subsequentuse.

Once the location and length of ad pods has been estimated by thesystem, the system 100 is able to use the estimated ad pod times toimprove the accuracy of various viewership calculations. At a block 216,the system uses the ad pod information to more accurately estimate thenumber of viewers viewing advertisements and more accurately estimatethe number of viewers viewing the content without the advertisements.For example, if the system has determined using the aforementionedtechniques that a program broadcast contains three 1 minute long ad podsbeginning at 9:03 PM, 9:13 PM, and 9:24 PM, then the system can estimatethe number of viewers watching those advertisements by calculating theaverage number of screens displaying the broadcast during the timeperiods when the ad pods were being played. Using the example viewershipdata in FIG. 3A, the system may calculate the average number of screensdisplaying the ad pods between 9:03-9:04 pm, 9:13-9:14 pm, and 9:24-9:25pm as 51,000 screens. The system may also calculate the average numberof viewers watching the overall broadcast 9:00 pm and 9:30 pm byaveraging the number of screens displaying the broadcast during thatperiod and arriving at 53,000 screens. The average number of viewerswatching the non-advertisement content of the broadcast may then bedetermined either by looking at the data for the time periods of theprogram broadcast that exclude the ad pods (i.e., 9:00-9:03 pm,9:04-9:13 pm, 9:14-9:24 pm, and 9:25-9:30 pm) or alternately by solvinga weighted average equation, which is based on the duration of theadvertisements, the duration of the overall broadcast, the averagenumber of screens of the overall broadcast, and the average number ofscreens for the overall broadcast. Namely:ad_portion=(3 minutes/30 minutes)=10%combined_screens=(ad_portion%*ad_screens)+((100%−ad_portion)*non_ad_screens)53,000=(10%*51,000)+(90%*non_ad_screens)non_ad_screens=(53,000−5,100)/90%non_ad_screens=53,222The calculated average number of viewers watching the non-advertisementcontent of the broadcast may provide a better estimate of the true reachof analyzed content as compared to the average number of viewerswatching the overall broadcast. Conversely, advertisers may not care asmuch about the estimated viewership of the non-advertising content asthey do about the advertising content, and may treat the estimate of theadvertisement content as a better measurement of the program or contentin question.

At a block 218, the system may convert the average number of viewerswatching the non-advertisement content into a viewership index thatallows advertisers and content producers to more quickly compare theperformance of one piece of content against another. To calculate theviewership index, the system calculates a proportion of the number ofscreens tuned to advertisements compared to the number of screens tunedto the overall broadcast. Using the previous example, the calculatedproportion is reflected as:(51,000/53,000)=96.2%The system might also calculate a proportion of the number of screenswatching advertisements compared to the number of screens watchingnon-advertised content:(51,000/53,222)=95.8%The calculated proportions may be scaled to derive indices that can bequickly compared across content. For example an “ad retention index” maybe determined by multiplying the proportion of screens displayingadvertisements compared to the number of screens watching the fullcontent by 100 (e.g., 96.2%*100=96.2 ad retention index). The system mayperform any determination, calculation, or analysis interchangeablybetween numbers representing screens, viewers, proportions of screens,and proportions of viewers.

It will be appreciated that the results of the analysis performed by thesystem 100 may be segmented by time, geography, demographic group, andthe like. For example, an analysis of viewing information associatedwith a first geographic region (e.g., “Washington region”) may becompared with viewing information associated with a second geographicregion (e.g., “Oregon region”) and a third geographic region (e.g.“Northwest region, including Washington and Oregon”).

Using techniques similar to the aforementioned analysis techniques, thesystem 100 may also calculate the number of screens for individualadvertisements in an ad pod rather than for the ad pod in aggregate. Forexample, if the starting and ending times of a 2 minute ad pod areknown, and if it is assumed that the ad pod consists of four contiguous30 second long advertisements, then the average number of screensdisplaying each of the four 30 second long advertisements can bedetermined by the system from the “number of screens” data.

In some embodiments, the system may employ data smoothing, such astechniques based on cubic or higher-order splines, in order to identifypoints of negative time derivative more accurately. It will beappreciated by one skilled in the art that there are many types of dataconditioning steps that may be used at various stages of the methoddisclosed herein. In some embodiments, the system may give more weightto certain events that cause a lost viewer as compared to other eventsthat cause a lost viewer. For example, a viewer who was lost from theaudience of a particular channel as a result of changing the channel toan adjacent channel may be given greater weight than a user lost as aresult of powering off their viewing device. Furthermore, the behaviorsof certain users who may be historically determined to be more likely tochange the channel as a result of a commercial (“channel surfers”), maybe given more weight for determining the starting or ending of an ad podthan other users not historically determined to frequently change thechannel. In addition, other types of supplementary viewing data, such astime-shifted data from a DVR device, may be used to identify the startor, especially, the end of an ad pod. For example, when a viewer isfast-forwarding through the ads using a DVR, he or she can see when theads end and use the end of the ads as a signal to resume normal-speedplayback. As a result information from a DVR on when users typicallyresume normal-speed playback may be used to more accurately estimate theend of ad pods.

Referring now to FIG. 4, a graphical representation of programviewership over a period of approximately five hours is shown. The graphrepresents analyzed tune data for a live content broadcast on a singlechannel over the graphed time frame. In FIG. 4, the live content is theSuper Bowl, a popular sporting event having significant viewership andsemi-scheduled advertisements.

In FIG. 4, the y-axis represents the viewing index, e.g., the adretention index, for the live content being analyzed. As previouslymentioned, the ad retention index can be calculated from the percentageof viewers tuned to a particular channel both during and not during theadvertisement portions over a predetermined time frame. The x-axisrepresents the predetermined time frame over which the tune data for aparticular channel has been collected. The measured viewing index, orad-retention index, for the live content over the five hour timeframe ofthe broadcast content is represented by line 500. The line 500 includesseveral drops and has a gradual increase over the course of thebroadcast. This corresponds to a traditional pattern in live sportingevents where more viewers “tune-in” to the end of the game to determinewho wins. Various events are readily discernible from the data. Forexample, the relatively flat portion of the line 500 in section 512 isindicative of half-time, where there is generally little change inviewers due to an extended break from game that allows viewers to doother activities.

Each vertical line, such as 506 and 508, denotes an ad-pod identified bythe system 100. Each ad-pod can vary in length and can vary from theoriginally scheduled time, due to changes in the live content beingbroadcast. However, as shown on the graph in FIG. 4, there is a strongcorrelation between the decline in viewership, or drops, and thedetected start of ad pods

The system 100 described herein is particularly useful for embodimentssuch as those described with reference to FIG. 4. With typical contentbroadcasts, advertisements are presented by a content provider on aregular and predictable basis. However, during live broadcasts,scheduled programming can vary. For example, during the Super Bowl aportion of the game may be extended to provide coverage of a touchdown.Accordingly, the scheduled advertisements may be delayed. The viewerbehavior system 100 can accurately detect the ad pod start and endingand therefore provide more accurate viewership assessments during thebroadcast content. Oftentimes, the aforementioned type of viewershipinformation is more valuable to advertisers to enable them to betterunderstand the reach of their advertisements and better direct theiradvertising budgets to certain content over others.

FIG. 5 depicts a representative report 600 that is generated by theviewing behavior analysis system 100. The report can be generated for aselected time frame, such as four (4) hours of content on a particularchannel. In other embodiments, the report can be generated by the systemon a per program, or specific content, basis. By virtue of the system100 including data from DVR playback devices, and other devices capableof streaming the content at times other than the allotted time frame atwhich the specified content is broadcast, a report generated on a perprogram basis is typically generated at a certain time after theoriginal content broadcast. Introducing such a delay to the reportgeneration allows the DVR and subsequent playback tune data to beincluded in the analysis. In other embodiments, the report can begenerated on a screen-type basis. For example, a report can be generatedbased on standard STB tune data, or based on streamed content, e.g.,through a DVR, DMR or Smartphone.

The report provides an interested parties, such as a content provider124, advertiser, content producer, etc., with comprehensive dataregarding viewer behavior during a particular program. For example, asshown in FIG. 5, the report 600 may include summarized informationregarding the viewer tune data and the program. A title 602 of thereport identifies the content being analyzed as “Program A.” A summaryregion 612 of the report provides various data characterizing thebroadcast and subsequent viewing of the program. For example, Program Awas broadcast over a fifty-eight (58) minute time frame on Jan. 25,2010. The content provider that broadcast the program had six (6) adpods scheduled during the programming and six (6) ad pods were detectedby the system 100. The summary region 612 can also provide the types ofplatforms on which the tune data was monitored, such as DVR andstreaming, along with the number of viewers, or screens, determined atthe beginning and end of the broadcast.

As shown in FIG. 5, each ad pod 604, 606 can be separately displayed inthe report 600 and summary information 614 provided about each add pod.Various measurements, such max viewers lost, loss rate, retention index,viewers returned, and comments, can be provided by the system in thesummary information 614. The system may also include graphicalrepresentations 608, 610 in the report 600 for each of the ad podsdetected during the monitored program. In some embodiments, the reportjust includes the ad retention index that is calculated for each ad podduring a program being monitored. In other embodiments, variousadditional measurements can be determined for each individual ad pod,such as loss rate for each increment over the duration of the ad pod.The aforementioned information can be useful, for example, foradvertisers to assess the performance of a specific advertisement duringa specific ad pod.

The report may also provide an indication (e.g., through highlighting orother visual notification) where content or ads had drops that occurredmore or less frequently. For example, the report may provide a statisticregarding a mean drop rate for the ad pods during the presentationperiod of specific content, or a mean loss of viewers during ad podsversus the overall mean loss of viewers for the content. The lower theoverall drop rate, the less frequently viewers were lost during thatpresentation period. As a specific example, the report may indicate thata third ad pod in an hour (60 min) of content broadcast at 10:00 amexperiences the most significant drop out of three (3) ad pods regularlyscheduled during that corresponding presentation period. Accordingly,the third ad pod may be highlighted red to indicate that the mostviewers/screens were lost and that advertisers may wish to avoid placingtheir ads in that particular ad pod. In addition, the report mayindicate that out of the three (3) aforementioned ad pods, the first adpod experiences the overall slowest rate of change, i.e., the mostgradual decline in viewers, and that the rate of change increases (e.g.,accelerates) the least over that ad pod. The rate of change andacceleration of an ad pod may indicate that an ad shown during that adpod, especially during the start of that ad pod where the maximum numberof viewers are still present, is much more likely to be viewed than atother times during the presentation period.

The report may also include a comparison of ad pod performance acrossmultiple presentations of similar content. For example, the system maycalculate viewing behavior associated with a sitcom program presentedacross multiple weeks. Using the statistics calculated each week, thesystem may include average statistics for ad pod presentation periodsand/or content presentation periods in the report. Providing averagestatistics across multiple period allows a user to compare statisticsfor a current program with historical averages to determine whether thecurrent presentation is better or worse than the norm.

The report can be generated by the system 100 and provided to aninterested party in various formats. For example, the report can be aweb page or web site, a digital image file, a PowerPoint presentationfile, a document for word processing software, a portable documentformat (“PDF”) file, a web browser readable html file, a spreadsheet, adatabase file, a communication format that is readable by a facsimilemachine, format consumable by a printer, and so on. Various additionaltechniques may be employed to generate and deliver reports, as describedfurther in U.S. patent application Ser. No. 13/096,964, filed Apr. 28,2011, entitled “METHOD AND SYSTEM FOR PROGRAM PRESENTATION ANALYSIS,”the contents of which are hereby incorporated by reference in theirentireties.

In alternative embodiments, the system may be implemented for audiomedia. For example, the system may analyze tuning data indicating whenend users are listening to audio content (such as a radio broadcast) andwhen they have stopped listing to audio content. In this embodiment, theaforementioned system and methods are applied to tune data reflecting auser's listening to audio. Such audio broadcasts may include satelliteradio, internet radio, over the air radio broadcast, as some examples.

Those skilled in the art will appreciate that the actual implementationof the data storage area may take a variety of forms, and the phrase“database” is used herein in the generic sense to refer to any area thatallows data to be stored in a structured and accessible fashion usingsuch applications or constructs as relational databases, tables, linkedlists, arrays, flat files, and so on. Those skilled in the art willfurther appreciate that the depicted flow charts may be altered in avariety of ways. For example, the order of the steps may be rearranged,steps may be performed in parallel, steps may be omitted, or other stepsmay be included.

From the foregoing, it will be appreciated that specific embodiments ofthe invention have been described herein for purposes of illustration,but that various modifications may be made without deviating from thescope of the invention. Accordingly, the invention is not limited exceptas by the appended claims.

We claim:
 1. A computer-implemented method performed at a viewingbehavior analysis system in communication across a network with aplurality of content providers and a plurality of video playbackdevices, the method for assessing drops in viewership during one or moreadvertisements that are broadcast in association with content having ascheduled programming variation, the method comprising: receiving, atthe viewing behavior analysis system, tune data across the network fromone or more clients, each client associated with a video playback deviceof the plurality of playback devices and capturing events representingviewing behavior of content on the associated video playback device;receiving, at the viewing behavior analysis system, a selection ofcontent from a user on which to perform a viewing analysis, theselection including the variation from scheduled programming; analyzing,at the viewing behavior analysis system, tune data associated with theselected content in order to construct a profile of viewership across anumber of clients, the viewership profile reflecting an actual orextrapolated number of viewers that viewed the content during acorresponding presentation period, the presentation period including thevariation from scheduled programming; analyzing, at the viewing behavioranalysis system, the viewership profile to identify one or more drops inviewers during the corresponding presentation period that exceeds athreshold rate, the identified one or more drops indicative of anadvertisement or advertisements presented in conjunction with thecontent; for each of the identified drops that exceeds the thresholdrate, estimating, at the viewing behavior analysis system, a start pointand an end point for the identified drop; calculating, at the viewingbehavior analysis system, a viewership loss rate for each drop, whereinthe loss rate is calculated over a time interval determined by theestimated start point and end point and wherein the loss rate reflects adrop in viewership as a result of the advertisements; and providing,from the viewing behavior analysis system, a report of the viewershiploss rate for each drop, the report indicating when the advertisementswere offset in response to the variation from scheduled programming in atime series graphical representation.
 2. The method of claim 1, whereinthe tune data includes power-up or power-down events, channel changingor content switching events, or playback events.
 3. The method of claim1, wherein the clients include a set-top box client, a digital videorecorder client, a viewer measurement device client, and a computingdevice client.
 4. The method of claim 1, wherein the content is selectedby the user by specifying a time period during which content waspresented.
 5. The method of claim 1, wherein the content is selected bythe user specifying a program or other discrete piece of content.
 6. Themethod of claim 1, wherein the extrapolated number of viewers isdetermined by taking an actual number of viewers and estimating a numberof viewers for which tune data is missing from clients.
 7. The method ofclaim 1, further comprising adjusting the actual or extrapolated numberof viewers based on a confidence in the tune data received from certainclients.
 8. The method of claim 7, wherein the adjustment is made byapplying a weighting factor to tune data from one or more clients. 9.The method of claim 1, wherein the threshold rate is based on apercentage of the tune data and the number of viewers at the start ofthe presentation period.
 10. The method of claim 1, further comprising:analyzing the identified drops to determine if a start point and an endpoint of different adjacent drops overlap in time; and applying areconciliation process for those identified adjacent drops that haveoverlapping start points and end points.
 11. The method of claim 1,further comprising using the calculated viewership loss rate for eachdrop to calculate an ad retention index for the corresponding content.12. The method of claim 1, further comprising: receiving a desiredgeographic or demographic segmentation; and filtering the analyzed tunedata associated with the selected content to only use tune dataassociated with the desired geographic or demographic segmentation. 13.A non-transitory computer readable medium encoded with instructionsthat, when executed by a processor, perform a method for assessing dropsin viewership during one or more advertisements that are broadcast inassociation with content having a scheduled programming variation, themethod comprising: receiving, at a viewing behavior analysis system,tune data across a network from one or more clients, each clientassociated with a video playback device of a plurality of playbackdevices and capturing events representing viewing behavior of content onthe associated video playback device; receiving, at the viewing behavioranalysis system, a selection of content from a user on which to performa viewing analysis, the selection including a variation from scheduledprogramming; analyzing, at the viewing behavior analysis system, tunedata associated with the selected content in order to construct aprofile of viewership across a number of clients, the viewership profilereflecting an actual or extrapolated number of viewers that viewed thecontent during a corresponding presentation period, the presentationperiod including the variation from scheduled programming; analyzing, atthe viewing behavior analysis system, the viewership profile to identifyone or more drops in viewers during the corresponding presentationperiod that exceeds a threshold rate, the identified one or more dropsindicative of an advertisement or advertisements presented inconjunction with the content; for each of the identified drops thatexceeds the threshold rate, estimating, at the viewing behavior analysissystem, a start point and an end point for the identified drop;calculating, at the viewing behavior analysis system, a viewership lossrate for each drop, wherein the loss rate is calculated over a timeinterval determined by the estimated start point and end point andwherein the loss rate reflects a drop in viewership as a result of theadvertisements; and providing, from the viewing behavior analysissystem, a report of the viewership loss rate for each drop, the reportindicating when the advertisements were offset in response to thevariation from scheduled programming in a time series graphicalrepresentation.
 14. The non-transitory computer readable medium of claim13, wherein the tune data includes power-up or power-down events,channel changing or content switching events, or playback events. 15.The non-transitory computer readable medium of claim 13, wherein theextrapolated number of viewers is determined by taking an actual numberof viewers and estimating a number of viewers for which tune data ismissing from clients.
 16. The non-transitory computer readable medium ofclaim 13, further comprising: adjusting the actual or extrapolatednumber of viewers based on a confidence in the tune data received fromcertain clients.
 17. The non-transitory computer readable medium ofclaim 15, wherein the adjustment is made by applying a weighting factorto tune data from one or more clients.
 18. The non-transitory computerreadable medium of claim 13, wherein the threshold rate is based on apercentage of the tune data and the number of viewers at the start ofthe presentation period.
 19. The non-transitory computer readable mediumof claim 13, further comprising: receiving a desired geographic ordemographic segmentation; and filtering the analyzed tune dataassociated with the selected content to only use tune data associatedwith the desired geographic or demographic segmentation.
 20. Thenon-transitory computer readable medium of claim 13, further comprising:using the calculated viewership loss rate for each drop to calculate anad retention index for the corresponding content.